Search for: All records

Coreset is a small set that provides a data summary for a large dataset, such that training solely on the small set achieves competitive performance compared with a large dataset. In rehearsal-based continual learning, the coreset is typically used in the memory replay buffer to stand for representative samples in previous tasks, and the coreset selection procedure is typically formulated as a bilevel problem. However, the typical bilevel formulation for coreset selection explicitly performs optimization over discrete decision variables with greedy search, which is computationally expensive. Several works consider other formulations to address this issue, but they ignore the nested nature of bilevel optimization problems and may not solve the bilevel coreset selection problem accurately. To address these issues, we propose a new bilevel formulation, where the inner problem tries to find a model which minimizes the expected training error sampled from a given probability distribution, and the outer problem aims to learn the probability distribution with approximately $$K$$ (coreset size) nonzero entries such that learned model in the inner problem minimizes the training error over the whole data. To ensure the learned probability has approximately $$K$$ nonzero entries, we introduce a novel regularizer based on the smoothed top-$$K$$ loss in the upper problem. We design a new optimization algorithm that provably converges to the $$\epsilon$$-stationary point with $$O(1/\epsilon^4)$$ computational complexity. We conduct extensive experiments in various settings in continual learning, including balanced data, imbalanced data, and label noise, to show that our proposed formulation and new algorithm significantly outperform competitive baselines. From bilevel optimization point of view, our algorithm significantly improves the vanilla greedy coreset selection method in terms of running time on continual learning benchmark datasets. The code is available at \url{https://github.com/MingruiLiu-ML-Lab/Bilevel-Coreset-Selection-via-Regularization}. 

Abstract Uranium (U) is an important global energy resource and a redox sensitive trace element that reflects changing environmental conditions and geochemical cycling. The redox evolution of U mineral chemistry can be interrogated to understand the formation and distribution of U deposits and the redox processes involved in U geochemistry throughout Earth history. In this study, geochemical modeling using thermodynamic data, and mineral chemistry network analysis are used to investigate U geochemistry and deposition through time. The number of U⁶⁺mineral localities surpasses the number of U⁴⁺mineral localities in the Paleoproterozoic. Moreover, the number of sedimentary U⁶⁺mineral localities increases earlier in the Phanerozoic than the number of U⁴⁺sedimentary mineral localities, likely due to the necessity of sufficient sedimentary organic matter to reduce U⁶⁺–U⁴⁺. Indeed, modeling calculations indicate that increased oxidative weathering due to surface oxygenation limited U⁴⁺uraninite (UO₂) formation from weathered granite and basalt. Louvain network community detection shows that U⁶⁺forms minerals with many more shared elements and redox states than U⁴⁺. The range of weighted Mineral Element Electronegativity Coefficient of Variation (wMEE_CV) values of U⁶⁺minerals increases through time, particularly during the Phanerozoic. Conversely, the range of wMEE_CVvalues of U⁴⁺minerals is consistent through time due to the relative abundance of uraninite, coffinite, and brannerite. The late oxidation and formation of U⁶⁺minerals compared to S⁶⁺minerals illustrates the importance of the development of land plants, organic matter deposition, and redox‐controlled U deposition from ground water in continental sediments during this time‐period.